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Abstract 

Our mission of capstone computing courses for the past ten years has been to offer students 
experience with the development of real-world information technology projects. This experience has 
included both the hard and soft skills required for the work they could expect as industrial 
practitioners. Hard skills entail extending one's knowledge structure with technical know-how, 
specifically using the latest software and hardware tools for building applications of genuine utility. 
Soft skills include the ability to work in a collaborative setting (e.g., to participate in team coordination 
and governance), the ability to interact with a customer (e.g., to establish product requirements and 
achieve acceptance), the ethos of creating value, and a facility for technical communications (written, 
oral, and electronic). Significant changes in the instructional environment have taken place in the ten 
years since the capstone class was first offered. This paper describes the adaptation to changes in the 
course's delivery so that its mission continues to be fulfilled successfully. 

Keywords: capstone computing courses, project-oriented courses, distance education, collaborative 
and teamwork skills, online student assessment 


1. INTRODUCTION 

The aim of our capstone is to familiarize 
students with how their trade is plied in 
organizations, so that the master program 
delivers "the practice" part of the promised 
"theory and practice." The projects are "real 
world" in every respect. They entail the 
development of an application desired by a real 
world customer. As in industry, applications are 
developed by a small, collaborative team which 
needs to communicate with the customer, 
coordinate its activity, attend to internal 
decision-making, and, as observed by Denning 
and Dunham (2001), be sensitive to delivering 


value. The applications press into service 
current technology. This is technology with 
which the students are usually unacquainted 
inasmuch as it may be specialized, new, or at 
least new to them. Students learn about real- 
world technology through their own group's 
experiences as well as through reports from 
other groups. A soft skill of transcending 
importance, emphasized by activities throughout 
the capstone, is the ability to communicate on 
technical concepts and issues; orally, in written 
reports, and via Web media; to peers and lay 
people. 

Capstone courses that provide real-world 
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projects for actual customers are not new. They 
are available in one or two-semester courses at 
both the graduate and undergraduate levels. 
Novitzki (2001), in describing a one-semester 
graduate course, focused on the administrative 
issues and found that the most consistent 
shortcomings of the students related to their 
working with functional managers, their group 
skills, and their communication skills. Two 
papers (Gorka, Miller, & Howe, 2007; Green, 
2003) described one-semester undergraduate 
courses that provided projects in conjunction 
with industry. Goold (2003) described how a 
one-semester undergraduate course evolved 
from small student teams of 4-5 students to 
relatively large teams of 10-12 students. Bruhn 
& Camp (2004) described a two-semester 
undergraduate course that required the full two 
semesters to provide an in-depth coverage of 
the phases of the systems development life 
cycle. A series of papers has described real- 
world information technology projects in 
masters-level capstone computing courses 
(Tappert & Cha, 2004; Tappert, Stix, & Cha, 
2007; Tappert & Stix, 2009; Tappert & Stix, 
2011 ). 

In the ten years since the capstone class 
assumed its project-based form, the most 
significant change has been in its presentation. 
In 2001-2002 the class spanned the fall and 
spring semesters and was face-to-face. In 2006 
it was condensed into a one semester offering. 
For projects, this meant that requirements 
elicitation, building the application, and the 
testing regimen were accelerated. We 
responded with agile methodology. In 2006 the 
class's delivery also shifted from face-to-face to 
"hybrid": online but with a meeting at the 

beginning of the semester for orientation, a 
meeting at the middle of the semester for team 
reports to the class, and a meeting at the end of 
the semester for final system presentation. By 
2009 the format was entirely online for portions 
of the class for whom attendance was 
geographically infeasible. This included 15 
students taking the class from India. 

What has not changed over time is the essence 
of the course. Groups are still required to 
maintain a Website for project tracking, have a 
single spokesperson for interacting with the 
customer, and attend to the division of labor. 
Projects must still be delivered. And a 
professional paper, about the project, must still 
be written by the group and presented at our 
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annual internal conference that provides 
students and faculty with the opportunity to 
present their research and project work. 

The remainder of this report goes into the 
details of each aspect of the course touched 
upon above. It explains how the course is 
currently managed and presents a 
comprehensive review of the projects completed 
over the past ten years. 

2. TEAM-ORIENTED CAPSTONE COURSE 

We use team projects modeled on real-world 
development practice to provide students with 
the educational experience of collaborative 
efforts, similar to what is done in industry, in 
order to design, build, and test computer 
information systems. We also discuss the 
pedagogical issues of managing information 
technology development projects conducted by 
geographically distributed student teams in an 
online course. 

Effective teamwork requires the division of 
responsibility, the coordination of efforts, 
communications to expedite coordination, and 
group governance for collective decision making, 
conflict resolution, and the control of deviance. 
Denning and Reihle (2009) draw attention to 
both the importance of group dynamics to 
software engineering and the traditional failure 
to accord them proper regard in project 
development courses. To pique the interest of 
students in "teamwork dexterity," which is even 
more critical to the functioning of distributed 
teams, we are capitalizing upon their 
enthusiasm for the television reality game shows 
such as Survivor and The Apprentice. 
Individuals in groups (tribes or teams) on these 
reality shows, as in the course, are: working 
toward common goals; acquiring and sharing 
new knowledge about the problem, the solution, 
and cooperative processes; harnessing the 
different skills of the different teammates; 
adjusting to the different personalities of the 
different teammates; exhibiting initiative but 
without disruptiveness; and learning to shoulder 
group obligations responsibly. The settings 
differ in significant interpersonal ways as well. 
For example, our project students don't get 
eliminated from the course, as participants can 
be eliminated from the game shows - like "voted 
out" on Survivor or "you're fired" on The 
Apprentice. Other differences are that reality 
show participants compete against each other, 
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competitiveness is encouraged, and devious 
behavior on the part of participants against 
other participants is accepted as part of the 
game. 

Beginning with the Fall 2006 semester, we 
migrated our highly successful, project-centered 
class from a traditional face-to-face format to an 
online format. While we had found mechanisms 
for overcoming the challenges that threatened 
the effective governance and achievement of 
traditional student development teams, in 2006 
we were confronting uncertainties about how 
these mechanisms port to teams working in the 
context of an online class and the new 
mechanisms that might need to be created. The 
online format precludes automatic, weekly 
assemblages that act as a safety net to the 
teams' interaction and smooth functioning. 

As the ability for impromptu team discussions 
before and after class disappeared and online 
communication became dominant, the internal 
dynamics of the development teams became 
more complicated. In addition, we needed to 
revisit the way we graded the performance of 
team members (see section 6). 

It is well known that projects undertaken by 
groups lacking co-presence presuppose a higher 
level of organizational and process skills among 
their members (Cusumano, 2008). The present 
paper describes procedures that enabled the 
successful functioning of student development 
teams in a largely online course. 

3. PROJECT-ORIENTED CAPSTONE COURSE 

The current capstone course is a project- 
oriented, one-semester, web-assisted course for 
masters-level computing students in which 
student teams develop real-world computer 
information systems for actual customers. 
Students learn the importance of a systematic 
approach in the process of developing robust 
systems, the management of projects, how to 
interact with customers and conduct 
requirements analysis, how to build and test 
systems, and the related technical and soft 
skills. Emphasis is placed on developing skills 
and knowledge in technical areas that have 
practical value in the workplace. In addition to 
technical skills, students develop problem¬ 
solving, critical thinking, communication, and 
teamwork skills. By working on real-world 
systems with actual customers, the students 
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learn the appropriate skills - both technical and 
soft skills - for filling meaningful roles in the 
professional IT workplace. 

Team Project Categories and Publications 

The team project focuses on developing a 
computer information system that meets an 
actual customer's real needs. Although the 
requirements for the projects come from the 
customers, the course instructor is the "boss" or 
"Chief Information Officer" of each project team, 
and, as such, the person who makes all the 
major decisions. The project customer knows 
what he/she wants as an outcome but may not 
know the technical aspects of the project work 
(algorithms, program code, etc.). Some 
projects have subject matter experts who are 
knowledgeable about certain domain related 
aspects of a project. The customer, the subject 
matter experts, and the instructor can give 
advice to help guide the teamwork but are not 
expected to make major contributions to the 
actual project development effort. 

Table 1 presents the 102 projects conducted 
over the last ten years together with the 
resulting 185 publications. Table 2 lists the 
project sources, Table 3 the publication 
categories, and the Appendix provides a detailed 
list of the publications. Of the 185 resulting 
publications, 142 were directly project-related, 
and 43 were similar in kind and designated 
"offshoot publications" (Table 1). 


Table 1. Summary of projects and publications. 


Project Category 

Number 

Projects 

Project 

Semesters 

Project 

Related 

Pubs 

Offshoot 

Pubs 

Web Applications 

21 

25 

21 


Pervasive Systems 

15 

25 

18 


PC Applications 

11 

18 

13 


Artificial Intelligence 

9 

11 

12 


Pattern Recognition 

9 

12 

34 

19 

Biometric Systems 

32 

35 

39 

19 

Quality Assurance 

5 

9 

5 

5 

Totals 

102 

135 

142 

43 


Table 2. Project sources. 


Project Source 

Number 

Faculty Ideas or Research 

42 

Student Ideas or Research 

36 

External Community 

13 

Internal University Needs 

11 

Totals 

102 
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Table 3. Publication categories. 


Publication Type 

Number 

External Conference Papers 

53 

Journal Articles 

7 

Book Chapters 

2 

Doctoral Dissertations 

17 

Masters Theses 

4 

Internal Conference Papers 

98 

Internal Technical Reports 

4 

Totals 

185 


Sample Projects and Websites 

In a recent semester we had seven projects as 
shown on the Projects page of the course 
website (Figure 1). Most of the project 
customers that semester were doctoral students 
enrolled in our Doctor of Professional Studies 
(DPS) program. The Projects page lists the 
projects and contains, for each project, the 
project ID number, the project customer(s) with 
links to detailed contact information (SME = 
Subject Matter Expert), a link to a detailed 
project description, and the student team (listing 
the team leader first). The project ID number is 
also a link to the student team website for the 
project. The team website for the "Keystroke 
Biometric" project is shown in Figure 2. 


Project Information 


ID Customer 

Project 

Student Team 

1 

Robb Zucker, DPS 

Dmitry Nikethimr. DPS 

Human Visual System 

Neural Network 

.Alexander Cipully 
Stamatio; Cheitdan; 
Roberto Rodriguez 

Rohit Yalnmanclu 

2 

John Stewart. DPS 

Stvlometrv System 

Edyta Zych 

Omar Canale; 

Yuuue Monaco 

Thouia; Morphy 


Sadia Uniat. DPS 

Alex Alexandrou, DPS 

Di NaravanMmthvtSMEl 

Biometric Pnxluet 

Investigation 

Juan.Ainadiz 

JiaTiauLm 

Giovanni Lngone; 
Slub'liankaTiipmanem 

4 

5 

6 

John Stewart. DPS 

Di Robert Zackl SME I 

Keystroke Biometric: 

Data Collection 

& System Testing 

Vuuuc Monaco 

Tyrone .Allman 

Mino Lanuabat 

Mandar Manohar 

Ned BakehnniL DPS 

Kevlogger Keystroke 

Biometric System 

Horace Heiuy 

John Dehica 

Pierre Folkw 

Dwight Worley 

Brenda Lvoiy 

Jack Freeman 

Jeiuiv Li. DPS < SME I 

Social Network 

Business Site 

Nancy Ratfaele 
Yogrta.Ahire 

Jenifer Neubaner 

7 

Steve Run. DPS 

Social Network 

Forensic Tools 

.Andrew Kambad 

YLdiul Aliucida 

Palak Shall 

David Wilkin; 


Figure 1. Project information on course 
website, spring 2011. 
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A continuing line of research, and one that 
brought forth many projects, is on the keystroke 
biometric, one of the less-studied behavioral 
biometrics. Keystroke biometric systems 
measure typing characteristics believed to be 
unique to an individual and difficult to duplicate. 
Over the last five years, long-text-input 
keystroke biometric systems for identification 
(one-of-n response) and for authentication 
(accept/reject response) have been developed. 
In this keystroke biometric area we have had 
about ten semesters of masters-level project 
work, four doctoral dissertations, three external 
conference papers, a book chapter, and a 
journal article. 



Figure 2. Example team website. 

Teams, Roles, and Methods of Work 

A team is a group of individuals having the 
responsibility to jointly accomplish an objective, 
and in this course the objective is to successfully 
complete a project. It is widely accepted that 
work in teams enhances learning by creating an 
"active learning process." Student teams have 
been found particularly effective when the 
students actually need each other to complete 
the project. It is also the norm for employees to 
work in teams, and teams are used in all kinds 
of organizations, such as in industry, education, 
and government. 

Most of the systems involve one or more of the 
following: programming, a database, a computer 
network, a Web interface. Java is the preferred 
language for projects that require programming. 
Non-programmers or weak programmers can 
contribute in many ways other than 
programming. A team usually consists of 3-5 
students - an Architect-Designer, one or two 
Implementers, a Quality Officer, and a team 
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Coordinator-Liaison. For small teams several 
team member functions can be combined. At 
least one team member, usually the 
Coordinator-Liaison, must be a good 
communicator for customer and instructor 
interactions. Once the project is underway, 
teams should interact at least once a week in 
addition to project work time, and interactions 
can be through a variety of communication 
modes, such as conference calls, email, online 
chat, comments affixed to work-related 
materials, and virtual or actual face-to-face. 

For project development work we use the agile 
methodology, particularly Extreme Programming 
(XP) which involves small releases and fast 
turnarounds in roughly two-week iterations 
(Beck, 2000). Each team delivers a prototype 
system that performs the basic required 
functions to their customer at the halfway point 
of the semester. This is possible since, 
according to the 80-20 rule (Pressman, 2010), 
80% of the project can be completed in 20% of 
the time it would take to deliver the complete 
system. A complete system is delivered at the 
end of the semester. 

4. PROJECT AND RESEARCH INTERPLAY 

Another aspect of this course is the interplay of 
student projects and research done by students 
and/or faculty. One of the novel approaches we 
use is to support student dissertation and faculty 
research to create research-supporting projects 
in several of our courses. We teach our 
dissertation students how to conduct research in 
a number of areas of computing, and our 
student project teams how to develop real-world 
computer information systems. In recent years, 
we have experimented with the interplay of 
dissertation research and projects created 
specifically to develop the supporting software 
infrastructure for that research. Some of the 
project customers are faculty members or 
dissertation students who need supporting 
software infrastructures to conduct their 
research. Thus, there is interplay between the 
project and research activities. 

We have found this interplay between research 
and project activities to be exciting and 
productive. The main benefits have been to 
increase faculty research productivity, to 
facilitate the completion of the doctorate 
program for gainfully employed information 
technologists, and to strengthen capstone 
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classes in the masters program. The mechanism 
has been using research problems to provide 
projects, and using projects to supply computing 
infrastructure. We term this symbiotic 
relationship the research/project interplay. 

The Doctor of Professional Studies in Computing 
program enables computing and information 
technology professionals to earn a doctorate in 
three years through part-time study while 
continuing in their professional careers. In 
contrast to project work which uses known 
technology to develop systems according to 
specified customer requirements, research is 
original, rigorous work that advances 
knowledge, improves professional practice, 
and/or contributes to the understanding of a 
subject. To graduate, each doctoral student is 
required to complete an original investigation 
presented as a dissertation. Masters students 
also have the option of a research thesis. 
Research methods depend upon the nature of 
the inquiry: controlled experiment, empirical 
studies, theoretical analyses, or other methods 
as appropriate. We require research work to be 
of sufficient strength to be able to distill from it 
a paper worthy of publication in a refereed 
journal or conference proceedings. 

5. COURSE MANAGEMENT 

Currently about two-thirds of the capstone 
students live or work in the greater NYC area. 
The remaining third come mostly from more 
distant regions of the east coast but some have 
been from as far away as California and Europe. 
Beginning in 2009 the course served cohorts 
from India - first a group from AOL and later a 
group from IBM. The distributed team issue is 
handled by a number of mechanisms and 
guidelines. 

To facilitate communication among the project 
stakeholders, we insist that, except for 
extenuating circumstances, communication 
between a team and instructor, and between a 
team and a customer, be through the team 
leader, with all team members copied on 
communication email and given summaries of 
face-to-face meetings. This reduces 

communication to the instructor from individual 
students and keeps all stakeholders updated on 
project activities. Although we had the same 
guideline when the course was conducted in the 
classroom with local students, this guideline is 
even more critical for distributed teams. Also, 
the instructor creates and uses email distribution 
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lists for the whole class, for each project team 
including the customer, and for all the 
customers. 

Project team leaders must be local, either living 
or working in the greater NYC area. This allows 
for easy communication and meetings between 
the project team leaders and the project 
customers, who have, so far, all been local. It 
also allows for similar contact between the 
project team leaders and the instructor, enabling 
the instructor to keep informed of the progress 
of the project work. 

The course website efficiently presents all the 
course information as described above for 
convenient centralized access. Most 

importantly, it contains the project-related 
information and links to the student-developed 
team project websites that are frequently 
updated with postings of project deliverables 
and other information. To ensure that the 
students read and understand the material on 
the course website, the first quiz contains 
questions on the course operation as described 
in the website material. 

The three 3-hour classroom meetings are 
important to bring the local students together so 
they can meet many of their teammates and 
form some face-to-face bonding. The first 
meeting occurs after the first week of the 
semester. By this time: 

• the students have introduced themselves 
online through a Blackboard forum, reviewed 
the course website, and submitted the 
project preference information form to the 
instructor 

• the instructor has received the students' 
project preferences and associated 
information, formed the student project 
teams, assigned teams to projects, chosen 
project team leaders, and posted the 
information on the project's page of the 
course website 

At this meeting the instructor and students 
introduce themselves face-to-face (half hour), 
the instructor gives a lecture on the nature and 
value of conducting real-world projects in a 
capstone course (one hour), the instructor 
reviews the specifics of the course material and 
describes each of the projects (one hour), and 
the students group themselves into their project 
teams and begin planning project activities (half 
hour). Some customers attend the first meeting 
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to introduce themselves and to meet the 
members of their team. 

At the second (midterm) meeting the students 
make PowerPoint slide presentations of their 
project prototypes. Material covered in these 
presentations includes, as appropriate and as 
time permits, a subset of the following items: 
brief description of project, summary of project 
specifications, frequency of meetings with 
customer/stake-holders and usual method of 
communication, plans to address changes in 
customer requirements, summary of user stories 
collected (if any), analyses accomplished 
(object-oriented might include defined classes 
and API's), design decisions and the trade-offs 
encountered, work breakdown structures, PERT 
chart, and/or Gantt chart, components 
built/planned, testing strategy, what was 
accomplished to complete the prototype, what 
will be added in the remainder of the semester, 
what has been easy/difficult during this half of 
the semester, and a prototype demonstration. 
Many customers attend the second meeting. 

At the third (semester-end) meeting the 
students present their final project system. This 
meeting is similar to the second meeting, and 
most of the customers attend the final 
presentations. 

Successful Teamwork at a Distance 

Although this is essentially an online course, we 
have three face-to-face meetings in a classroom 
during the semester: one near the beginning, 
one near the middle, and one at the end of the 
semester. These contacts, presence at which is 
highly recommended but not required, are 
typically attended by about two-thirds of the 
students - those who live or work in the greater 
New York City area. The first contact is 
important because it introduces communication 
standards and the archiving of course 
information. An extensive course website 
presents all the course information, with links in 
the left menu area providing access to the 
sections (pages) of the website: 

• Homepage - includes the instructor 
information, textbooks, course description 
and goals, course requirements, and grading 
system. 

• Syllabus - lists the weekly readings and 
assignments. 

• Projects - contains a table of the semester's 
projects, and provides for each project the 
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customer's name and contact information, 
the description of the project, the names of 
the students on the development team 
assigned to the project, and a link to the 
project team's website. 

• Students - contains photos of the students 
so students know their classmates and the 
instructor can recall a student (possibly 
years later) when providing a letter of 
recommendation. 

• Project Deliverables - lists and describes the 
project deliverables. 

• Grades - contains a table of the graded 
events and the current student grades 
indexed by the last 4 digits of their 
university ID number. 

• A link to the Blackboard educational 

software system (Blackboard, 2012) used for 
quizzes, discussions, and collecting digital 
assignments. 

The instructors solicit and interact with potential 
customers to set up new projects, work with the 
university computer support personnel to assure 
the presence of the required project 

development software and computing 
infrastructure, and monitor the systems' 

development process. Projects come from 
faculty and dissertation students interested in 
developing systems to further their research, 
from other departments or schools of the 
university needing computer information 

systems, from non-profit community institutions 
such as local hospitals, from local research 
institutions, and from interests of the project 
students. The instructor sizes and shapes each 
project to be an appropriate systems 
development experience for the students, forms 
the student teams, and assigns each team to a 
project. 

From the project descriptions posted on the 
course website the students complete a project 
preference form during the first two weeks of 
the course. They list their current company and 
job title, number of years of work experience in 
information technology, work and home 
locations, whether they can attend the three 
classroom meetings, preferred communication 
mode (email, phone, online chat, Skype, 
Facebook, Twitter, etc.), top five project choices, 
top five availability time choices for project 
communication (day of week plus morning, 
afternoon, or evening), project skills 

(requirements engineering, system design, 

programming, databases, web design, 

networking, communication/leadership, etc.). 
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The instructor uses this information to form 
teams, to select team leaders, and to assign 
teams to projects. 

Blackboard Educational Software 

The Blackboard educational software system 
(Blackboard, 2012) is used for quizzes, for 
collecting digital deliverables, and for discussion 
forums. There are discussion forums for 
archiving all instructor email to the whole class 
for easy reference, for student introductions 
(students are asked to introduce themselves 
online during the first week of the semester), for 
discussions related to the textbook and other 
course material, and for discussions relating to 
each of the projects. The project forums are 
used to discuss project-related material, and 
each project team is required to post a weekly 
project status report on their project forum. It 
might be mentioned that previously student 
teams gave their status reports verbally in the 
classroom and students could benefit by learning 
about the other projects and hearing the 
instructor feedback, whereas now they are 
posted on the project forums (and 
simultaneously on project websites) where they 
are less likely to be reviewed by students in 
other projects. 

6. STUDENT ASSESSMENT 

Student assessment is currently as follows: 
individual quizzes (20%), initial team 
assignment (10%), team project midterm 
(20%), team project final (20%), and team 
project technical paper (30%). Thus, 80% of a 
student's grade is based on their contribution to 
the team effort with the quizzes (based primarily 
on the textbook material) providing the only 
direct individual assessment. Mid-term and final 
exams used in a previous two-semester course 
were eliminated allowing the students to focus 
on the project work in this one-semester course. 
The team has the ultimate responsibility for the 
project work and is graded accordingly. Grades 
on team events are determined by first 
assigning a team grade and then adjusting an 
individual student's grade up or down based on 
evaluations of the student's contribution from 
the instructor, the project's customer(s), and the 
student's teammates. 

Because this is a project-oriented course with no 
midterm or final exams, student grades depend 
mostly on their contribution to the project work. 
The usual expected time commitment per 
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student for a 3-credit course is 3 hours per week 
in class and twice that outside of class, for a 
total of 9 hour per week. However, because this 
is an online course where students save 
commuting time, we expect a time commitment 
of about 10 hours per week, and this additional 
time commitment is one of the advantages of a 
distance-learning course. 

Self and Peer Evaluations 

Finally, we use peer evaluations to assess the 
project contributions of each team member. 
Although used when the course was conducted 
in the classroom, peer evaluations are even 
more critical for distributed teams because some 
team members have minimal, if any, direct 
contact with the customer and instructor. 
Obtaining individual student grades on 
teamwork has been reported in the literature. 
For example, Clark, Davies, & Skeers (2005) 
created an elaborate web-based system to 
record and track self and peer evaluations, 
Brown (1995) has a system similar to ours but 
which uses more granular numerical input, and 
Wilkins & Lawhead (2000) use survey 
instruments. 

The students are required to provide self and 
peer evaluations three times during the 
semester - once after the initial assignment 
primarily to acquaint the students with the 
process, at the midterm checkpoint, and at the 
end-of-term checkpoint. They evaluate each 
team member, including themselves, by 
assigning "=" for average contribution, "+" for 
above average contribution, and for below 
average contribution. Multiple "+" or signs 
can be used to indicate extra strong or extra 
weak contributions, but the total number of plus 
and minus signs the evaluator assigns must 
balance out (i.e., be equal in number). A team 
grade for a particular deliverable or time interval 
is first determined, and then grades for 
individual students are adjusted relative to the 
team grade based on the peer evaluations along 
with additional input from the customers and 
instructor. For example, a typical peer 
evaluation summary chart with associated 
grades is shown in Table 4 for a four-member 
team. Each of the four evaluation columns 
shows the evaluation of a team member 
evaluating him/herself and the other team 
members. The summary column shows the sum 
of each row of evaluations, and the grade 
column shows the student grades. Here, a team 
grade of 85% is first determined and then 


individual grades are adjusted relative to the 
team grade, in this case up or down 2% for each 
"+" or sign. For simplicity, this table shows 
only the peer evaluations, but customer and 
instructor evaluations are usually included as 
well. Team leader and instructor evaluations 
can be given extra weight, and self evaluations 
that appear overly inflated are usually 
eliminated. 


Table 4. Team peer evaluation and grade chart. 


Team 

Member 

Eval 

1 

Eval 

2 

Eval 

3 

Eval 

4 

Summary 

Grade 

1 

+ 

= 

+ 

+ + 

+ + + + 

93 

2 

= 

= 

- 


— 

79 

3 

- 

= 

+ 

- 

- 

83 

4 

= 

= 

- 

+ 

= 

85 

Average 

= 

= 

= 

= 

= 

85 


Students are also asked a number of general 
questions for the time interval in question - the 
number of hours per week spent on project 
work, their specific contributions, their strengths 
and how these were used, their areas needing 
improvement, and what has enhanced and/or 
handicapped their team's performance - and the 
responses might influence the instructor 
evaluation of a student's contribution to the 
team effort. For additional input the instructor 
can discuss team member contributions with the 
team leader. 

Customer Evaluations 

At the end of the semester we survey the 
students using the Survey Monkey (2012) web- 
based survey system to obtain feedback on the 
team-customer interactions during the 
semester: whether the customer's initial project 
specifications were clear and understood, 
whether the amount of contact/interaction was 
adequate, whether the speed of response to 
questions was adequate, and whether the 
continued guidance and direction on the project 
work was sufficient. This information is used to 
determine the team satisfaction with a customer 
and, for example, whether to continue or not 
continue a project with a particular customer. 

Pedagogical Evaluations 

At the end of the semester we survey the 
students to obtain feedback on the course 
methodologies and procedures, such as what 
has worked well or not well from the students' 
point of view. We use these pedagogical 
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evaluations to change our methodologies and 
procedures from time to time, and to keep 
informed on the technologies and methodologies 
the student teams are using. We find, for 
example, that student teams use many modes 
of communication. 

7. OVERALL BENEFITS 

There are many benefits of the research and 
project activities. The real-world projects 
provide valuable systems for the customers, 
allow the students to develop technical and 
value skills, utilize student-centered team 
learning, foster interdisciplinary collaboration, 
encourage student involvement in the university 
and local communities, support student and 
faculty research, and enhance relationships 
between the university and local technology 
companies. Overall, these projects result in a 
beneficial outcome for all concerned. 

A side benefit is the presentation and publication 
activities that enhance communication skills. 
We have both the research and project students 
produce papers for publication, which is a novel 
aspect of our teaching approach. For the 
dissertation student we encourage publication, 
even if only for an internal conference or 
workshop, soon after the student obtains 
preliminary results. Our yearly internal 
conference, complete with a review process and 
proceedings, is for this purpose. We have found 
this helpful because it is much easier to begin by 
writing a small paper than a large dissertation, it 
solidifies the problem statement and general 
approach with some preliminary results, it 
ensures that the student and advisor have a 
common understanding of the problem and 
methodology and that the advisor buys into the 
process, and it generates ideas and motivation 
for extending the work into a significant 
research study acceptable as a dissertation. We 
have found that working to produce publications 
is a strong motivating factor for the students. 
The publications also enhance the external 
image and identity of our programs. 

The various customers benefit from the systems 
created for them by the students, sometimes 
receiving systems they might not obtain under 
ordinary circumstances. The customers include 
the research students, the faculty, the internal 
and greater university communities, and the 
community non-profit and technology 
organizations. The work with other universities, 
such as the Rensselaer Polytechnic Institute, 
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extends our collaboration to the greater 
university community. The projects also extend 
into the local community, involving three local 
hospitals, the IBM speech and pen computing 
groups, and a small company, to provide the 
students with off-campus experiences and to 
foster an extended community for learning and 
growth. 

8. CONCLUSIONS 

The online course format necessarily means a 
reduction in the face-to-face contact time of 
student teams jointly working on projects 
inasmuch as weekly class assemblages no longer 
exist. All courses with a collaborative 
component requiring groups to complete a task 
requiring cooperation and coordination over an 
extended time will find that the students are 
forced into working in a distributed context. For 
projects' success, and therefore course success, 
effective techniques for managing distributed 
student teams are required. We confronted this 
pedagogical issue head-on in a masters-level, 
capstone course in which teams of students in 
computer science and internet technology 
develop real-world systems for actual 
customers. This course had been in successful 
operation for over five years in the face-to-face 
mode when it shifted to online. Flere we 
experienced success as well. 

Our success in the online mode rests on much of 
the same management infrastructure that had 
facilitated effective communications among 
"traditional teams," notably the website that 
comprehensively centralized access to project 
information and Blackboard for organizing digital 
deliverables and discussion forums. The new 
pedagogy consists of an initial face-to-face 
contact offering a rigorous introduction to the 
usage of the information dispensing and 
communication channels, the requirement that 
the team leader live locally and be amenable to 
in-person meetings with the customer and the 
instructor, and rigid requirements about 
circulating communications and archiving 
documents. 
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